1. Field of the Invention
The present invention relates to a motion vector detecting device, especially to a motion vector detecting device suitable for use in the inter-frame coding of the motion compensation being the compression coding system of dynamic image information.
2. Description of the Related Art
Dynamic images include many similar images in two continuous screens. To reuse the previous images of these similar images in two continuous screens will make it possible to reduce the transmission of information to form the images. That is, the reuse of the previous images will achieve the compression coding of the dynamic images. To achieve this requires information indicating the location in the present screen which a part of the previous screen moves to (namely, a motion vector).
The detection of the motion vector generally employs the following method. First, the method gazes a. block consisting of Mxc3x97N pixels (including M=N) of the present screen. Further, it gazes each of the blocks of Mxc3x97N pixels contained in a (M+xcex1)xc3x97(N+xcex2) pixel region (hereunder, referred to as a xe2x80x9creference regionxe2x80x9d, also including xcex1=xcex2) which is wider than the block consisting of the Mxc3x97N pixels of the previous screen (hereunder, referred to as the xe2x80x9creference screenxe2x80x9d). And, it searches the most similar block to the gazed block of the present screen among the blocks of the Mxc3x97N pixels contained in the reference region. Concretely, the sums of the differential absolute values are calculated between corresponding pixel data of the blocks of the present screen and the reference region. The block is searched in the reference region, which reduces the sum of the differential absolute values to the minimum. The motion vector is expressed by the difference (xcex94x, xcex94y) of the coordinates between the searched block and the gazed block of the present screen.
In practice, the detection of the motion vector is executed by a plurality of steps in order to enhance the processing efficiency or to increase the processing speed in many cases. The actual steps in an example are as follows.
First, the gazed block of the present screen and the reference region of the reference screen each are reduced, when a block of the present screen is compared with each of the blocks of the reference region.
The first method to reduce the foregoing block thins out every other pixel, for example, from the Mxc3x97N pixels into a block of M/2xc3x97N/2 pixels as to the block of the present screen, and thins out every other pixel from the (M+xcex1)xc3x97(N+xcex2) pixels into a block of (M+xcex1)/2xc3x97(N+xcex2)/2 pixels as to the reference region. And, the second method obtains the averages of four pixels each, for example, from the Mxc3x97N pixels to make a block of M/2xc3x97N/2 pixels as to the block of the present screen, and obtains the averages of the four pixels each from the (M+xcex1)xc3x97(N+xcex2) pixels to make a block of (M+xcex1)/2xc3x97(N+xcex2)/2 pixels as to the reference region.
Next, the second step gazes a part including the block that was searched in the first step, of the reference region not reduced; and it searches the most similar block to the block of the present screen from the part region gazed. Further, some cases need a third step and the succeeding steps (for example, the detection of the motion vector employing the reference region of the half pixel accuracy explained later).
However, the conventional motion vector detecting device involves the following problems in the improvement of the processing speed and/or the simplification of the device construction.
(1): The detection of the motion vector through a plurality of the foregoing steps has generally adopted the following procedure in the conventional technique. FIG. 36 is a conceptual chart to explain the procedure.
A motion vector detecting device 11 contains an internal memory (also called as a local memory) 11a and a motion vector detecting unit 11b. The internal memory 11a is generally made up with an SRAM.
The image data of the present screen and the reference screen are stored in an external memory 13 provided outside the motion vector detecting device 11. The external memory 13 is generally made up with a DRAM.
In the conventional method of the detection of the motion vector, the data of the Mxc3x97N pixels (for example, 16xc3x9716 pixels) are inputted as they are from the external memory 13 into the internal memory 11a, as an information of the present screen. Further, the data of the (M+xcex1)xc3x97(N+xcex2) pixels (for example, 48xc3x9748 pixels) are inputted as they are, as an information of the reference region. And, at the first step to execute the comparison between the reduced blocks, the motion vector detecting unit 11b reduces each of the blocks of the present screen and the reference region of the reference screen as mentioned above.
The reason that the Mxc3x97N pixels as the information of the present screen and the (M+xcex1)xc3x97(N+xcex2) pixels as the information of the reference region each have been inputted in the internal memory as they are mainly lies in the facility in the process of the second step and after, namely, the process of the comparison between the blocks not reduced.
However, this conventional method requires the storage capacity of at least Mxc3x97N+(M+xcex1)xc3x97(N+xcex2)+M/2xc3x97N/2+(M+xcex1)/2xc3x97(N+xcex2)/2 for the internal memory 11a, and therefore, a memory of a large storage capacity has been necessary for the internal memory.
(2): Further, in the conventional method, the data of the blocks of the present screen and the data of the reference region of the reference screen each are stored in the internal memory 11a from the external memory 13 in the state that they are not reduced, and thereafter, they are reduced. Therefore, the conventional method requires a time to store the data and a time to reduce the data separately, and needs a longer processing time.
(3): Further, in the conventional method, every time when the motion vector of one block of the present screen is being detected, the data of the reference region corresponding to the one block is transferred to the internal memory 11a. However, the reference regions corresponding to the adjoining blocks of the present screen have an overlapping region in many cases. As shown in FIG. 37, suppose that the present screen 21 contains first and second blocks 21a, 21b, each adjoining. The reference screen 23 contains first and second reference regions 23a, 23b corresponding to the first and second blocks 21a, 21b. These reference regions 23a, 23b often contain overlapping image data 25. In such a situation, the conventional method transfers this overlapping image data 25 to the internal memory 11a during detection of the motion vector of the first block 21a and during detection of the motion vector of the second block 21b. Accordingly, the conventional method requires a longer time to transfer the data.
(4): Further, the compression coding of dynamic images frequently employs inter-frame predictive coding. And generally, the judgment whether or not to execute the inter-frame predictive coding with regard to a block (also referred to as a gazed block) of the present screen is performed as follows. First, the sum of the differential absolute values between the average of all the pixel values of the gazed block and each of the pixel values of the gazed block is calculated (this is also referred to as a second sum of the differential absolute values). And, this second sum of the differential absolute values is compared with the sum of the differential absolute values (also referred to as a first sum of the differential absolute values) that was obtained for detecting the foregoing motion vector. From this comparison result, whether or not to execute the inter-frame predictive coding is judged.
However, the conventional method separately performs the operation to obtain the first sum of the differential absolute values and the operation to obtain the second sum of the differential absolute values. Accordingly, the conventional method takes a longer operation time.
(5): Further, the detection of the motion vector through a plurality of the steps requires a third step frequently, as described hereunder.
New pixel data are calculated on the basis of the pixel data of a part including the block of the reference region not reduced, which was searched by the foregoing second step, and the adjoining pixel data of the foregoing pixels. A new reference region (reference region of the half pixel accuracy) is generated by means of the new pixel data and the original pixel data. And, the most similar block to the foregoing gazed block is searched from the reference region of the half pixel accuracy. This third step is often carried out in the conventional method.
Accordingly, the motion vector detecting device that detects the motion vector by the third step requires a circuit to reduce each of the gazed block and the reference region, and a circuit to generate the foregoing reference region of the half pixel accuracy. However, to provide both the circuits individually will increase the hardware, or the circuit scale of the motion vector detecting device.
Therefore, the motion vector detecting device is desired which exceeds the conventional method in terms of the processing speed and the simplification of the device construction.
The object of the invention is to provide a motion vector detecting device capable of solving the foregoing problems.
(1) In order to accomplish the foregoing object, the motion vector detecting device according to one aspect of the invention detects the motion vector of a gazed block, by means of a process including a first step to reduce each of the gazed block of a present screen and a reference region of a reference screen and to search a block most similar to the reduced gazed block from the reduced reference region, and a second step to search a block most similar to the gazed block from a part including the block searched in the first step, of the reference region not reduced. And, the motion vector detecting device stores each of the gazed block and the reference region in the internal memory, and executes the searches in the first and second steps on the basis of the stored information. Further, the motion vector detecting device is characterized by comprising an image processing unit that reduces at least one of the gazed block and the reference region into a form that can be used in the first step, and thereafter, transfers the reduced one to the internal memory.
In the motion vector detecting device provided with this image processing unit, at least one of the gazed block and the reference region is reduced into a form that can be used in the first step, and thereafter, stored in the internal memory; and therefore, the capacity of the internal memory can be reduced to that extent.
Further, to implement the invention, preferably, the image processing unit is constructed such that at least one of the gazed block and the reference region is sequentially reduced, and the reduced one is sequentially transferred to the internal memory. With this construction, the reduction process and the storage process to the internal memory can be carried out substantially in parallel; and therefore, the processing time can be shortened to that extent.
Further, to implement the invention, both of the gazed block and the reference region may be reduced, or either one of them may be reduced. However, in case either one of them is reduced, preferably the reference region is reduced. The reason is that the reduction of the reference region gives a greater contribution to the capacity reduction of the internal memory, since the capacity of the reference region is larger than the capacity of the gazed block.
Further, to implement the invention, when the motion vector detecting device is a device to execute the third step that generates the foregoing reference region of the half pixel accuracy, and searches a block most similar to the gazed block from the reference region of the half pixel accuracy, preferably the image processing unit is used also as a means to generate the reference region of the half pixel accuracy.
With this arrangement, the circuit to reduce at least one of the gazed block and the reference region can be shared with the circuit to generate the reference region of the half pixel accuracy. Thereby, the circuit scale of the motion vector detecting device can be reduced.
Preferably, such an image processing unit is made up with a first means to add values of two horizontal pixels of an image to thereby generate a horizontal two pixel average, a second means to be selectively used as either a means to add values of two horizontal pixels of the image to thereby generate a horizontal two pixel average, or a means to add values of two vertical pixels to thereby generate a vertical two pixel average, and a third means to add a sum calculated by the second means and a sum calculated in the past by the second means to thereby generate a four pixel average.
Although the detail will be described later, the image processing unit of this preferred example is able to calculate the pixel value to obtain an image in which the image is reduced into 1/4 by the second means. Further, to use the firstxcx9cthe third means in combination enables the image processing unit to calculate the pixel values of the horizontal two pixel average, the vertical two pixel average, and the four pixel average for obtaining the reference region of the half pixel accuracy.
(2) Further, the motion vector detecting device according to another aspect of the invention searches a part most similar to a gazed block of a present screen from a reference region of a reference screen to detect a motion vector of the gazed block, in which the gazed block and the reference region (including a case in which the same are stored in a reduced state) are stored in an internal memory contained in the motion vector detecting device and the search is executed on the basis of the stored information. And, the motion vector detecting device is characterized by comprising a reference region preparation unit that rewrites only a part of the reference region stored in the internal memory into a new data in order to detect the motion vector of a previous gazed block, and thereby forms a reference region for a gazed block for this time in the internal memory.
As already explained with reference to FIG. 37, the reference regions for adjoining gazed blocks have an overlapping part in many cases. Therefore, the most parts of the reference regions that are stored in the internal memory to detect the motion vector of the previous gazed block can be considered as the common parts with the reference regions to detect the motion vector of the gazed block for this time. Accordingly, it will not conceivably give any problems to the detection of the motion vector to rewrite with new data only a part of the reference regions stored in the internal memory and use the new data as the data of the reference region for detecting the motion vector of the gazed block for this time. With this arrangement, the transfer data from the external memory to the internal memory can be reduced in quantity, and the processing speed of the motion vector detecting device can be increased.
Further, the extent of data to be rewritten into the new data out of the data of the reference regions stored in the internal memory will be determined in accordance with the design. Although it is not limited thereto, in the embodiment described later, the data of the reference regions stored in the internal memory are designed to be rewritten into new data by each 1/3.
(3) Further, the motion vector detecting device according to still another aspect of the invention detects the motion vector of the gazed block by means of a process including a first step to reduce each of a gazed block of a present screen and a reference region of a reference screen and search a block most similar to the reduced gazed block from the reduced reference region, and a second step to search a block most similar to the gazed block from a part including the block searched in the first step, of the reference region not reduced. And, the motion vector detecting device connected to an inter-frame predictive coding device in use comprises a differential absolute value summing calculation unit including a plurality of processor elements (PE) that calculates, for the searches in the first and second steps, a sum of the differential absolute values (the first sum of the differential absolute values) between pixel data of the gazed block and pixel data of the reference region. Further, the differential absolute value summing calculation unit uses a PE that is not in use for the calculation of the first sum of the differential absolute values during a period of the first step or the second step, and thereby calculates either one of an average value of all the pixels of the gazed block, which is used as a parameter for judging whether or not to execute an inter-frame predictive coding, and a sum of the differential absolute values (the second sum of the differential absolute values) between the average value and each pixel data of the gazed block.
When the motion vector is obtained through a plurality of steps, the number of processes to calculate the sum of the differential absolute values through the first step is larger than the number of processes to calculate the same through the second step and after. Accordingly, the number of PEs is set so that the processes in the first step can be carried out effectively. As a result, there emerge PEs that are not used for the subsequent processes, namely, vacant PEs in the second step and after. This invention utilizes the vacant PEs actively. Thereby, the motion vector detecting device of the invention is able to carry out, without increasing the circuit scale of the device, the process to detect the motion vector and the process to calculate the parameter for the judgment of the inter-frame predictive coding in parallel. Therefore, the motion vector detecting device will achieve a higher processing speed without increasing the circuit scale of the device.
Further, to implement the invention of the motion vector detecting device provided with the differential absolute value summing calculation unit in which the vacant PEs are utilized, when the motion vector detecting device is a device to execute the third step that calculates an average value of adjoining pixel data in a part including the block searched in the second step, of the reference region not reduced, to thereby generate a new pixel, generates a new reference region (reference region of a half pixel accuracy) with the new pixel and the original pixel, and searches a block most similar to the gazed block from the reference region of the half pixel accuracy, preferably, the differential absolute value summing calculation unit is the means as follows.
Preferably, the differential absolute value summing calculation unit uses a PE that is not in use for the calculation of the first sum of the differential absolute values during the period of the second step to thereby calculate the average value of all the pixels of the gazed block, and uses a PE that is not in use for the calculation of the first sum of the differential absolute values during the period of the third step to thereby calculate the second sum of the differential absolute values.
This arrangement necessarily enables the average value of all the pixels of the gazed block to be obtained. Then, the average value obtained can be used to calculate the second sum of the differential absolute values. Therefore, it becomes possible to effectively use the vacant PEs in accordance with the procedure, and the efficiency to detect the motion vector can be enhanced effectively.
(4) Further, to implement the invention, preferably, the motion vector detecting device is provided with at least two of the foregoing image processing unit, the reference region preparation unit, and the differential absolute value summing calculation unit. This arrangement will simplify the construction of the motion vector detecting device and increase the processing speed of the same more efficiently than a device provided with the foregoing units individually.
Further scope of applicability of the present invention will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art from this detailed description.